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Executive Summary 



Since the mid-1990s, the Pennsylvania Department of Edueation (PDE) has required all 
students eoneentrating in eareer and teehnieal edueation (CTE) programs to eomplete a 
standardized teehnieal skill assessment at or near the end of their program. Results of 
technical skill assessments are used for a number of purposes, including recognizing 
student aehievement, supporting program improvement and professional development 
efforts, and holding edueators aeeountable for their students’ performanee. Interest in 
sueh assessments and their use are inereasing nationwide, partieularly in response to the 
2006 Perkins Aet requirement for reporting on eareer and teehnieal skill proficieneies 
(Hyslop, 2009). Researeh, however, has yet to fully relate technical skill levels, as 
measured by high sehool graduates’ performanee on broad work readiness or narrow, 
ooeupation-speoifio teehnieal skill assessments, to their subsequent employment and/or 
postsecondary enrollment outcomes. 

To address this gap in the literature, this study analyzes data on over 21,500 high sehool 
graduates with a CTE eoneentration in Pennsylvania who eompleted a workplaee 
readiness or oeeupation-speeific assessment developed by the National Oecupational 
Competeney Testing Institute (NOCTI). To eontrol for student demographie 
eharaeteristies and edueational experiences, assessment seore reeord data for the 2005- 
2006 and 2006-2007 aeademie years were merged with student-level administrative 
reeords maintained by the Bureau of Career and Teehnieal Education, PDE. Eollow-up 
data on graduates’ post-program work experienees were obtained from a state survey 
administered to all CTE program graduates to aseertain their job status in the seeond 
quarter following their high school completion. Postseeondary enrollment data were 
obtained by matching student identifiers with records maintained by the National Student 
Clearinghouse (NSC) to determine whether individuals enrolled in a postseeondary 
institution at any time between eompleting high sehool and the fall of 2009. 

Deseriptive statistics identify the demographic, high school academic, and CTE program 
characteristics of graduates who took workplaee readiness and oceupation-speeifie 
teehnieal skill assessments and show graduates who had earned industry eertifieations by 
program type and skill level. The seeond half of the analysis estimates logistie regression 
models of the effect of technical skill level (as measured by a teehnieal skill assessment 
or industry-reeognized certifieation) on the odds of postseeondary enrollment following 
high sehool graduation. 

Although Pennsylvania is one of the most advanced states in administering teehnieal 
assessments to all seeondary CTE program graduates, assoeiating test performanee with 
post-program plaeements is eompromised by low response rates. Of the nearly 30,000 
graduates for whom NOCTI test seores were available, researehers were able to mateh 
edueational reeords for just over 21,500 individuals (72%). Of these, researchers were 
able to identify self-reported employment outcomes for nearly 6,100 (29%) and 
postseeondary enrollment outeomes for roughly 8,800 (41%) CTE graduates. Missing 
data were greatest for students seoring at the lowest level of the teehnieal assessments. 
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Among the graduates for whom reeord matehes were aehieved, deseriptive statisties 
indieate positive assoeiations between graduates’ teehnieal skill level as measured by a 
technieal skill assessment, being female, and enrolling in a Health Oeeupations or 
Oeeupational Home Eeonomics program. Skill levels also are positively assoeiated with 
graduates’ grade point average (GPA) and the number of advaneed math and seienee 
courses passed. ^ 

Multivariate analyses indicate that, when controlling for race/ethnicity, gender, the 
number of advanced science and math courses passed during high school, and GPA, the 
odds of GTE program graduates enrolling in higher education are greater by a factor of 
1.39 for those with the highest level of technical skill than those with the lowest level, as 
measured by performance on an occupation-specific assessment. Among GTE graduates 
who took a workplace readiness assessment, the odds of enrolling in postsecondary 
education among those scoring at the highest skill level are greater by a factor of 2.22 
than those scoring at the lowest skill level. Although data on industry-recognized 
certifications were more limited, no statistically significant relationship was found 
between earning a certification and enrolling in higher education. 



* Comparisons reported in the text were tested using Pearson’s chi-square test for independence and are 
significant at the /? < .05 level. 
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The Carl D. Perkins Career and Teehnical Edueation Improvement Act of 2006 (Perkins 
IV) mandates that states report on the technical skill attainment of students concentrating 
in career and technical education (CTE) using assessments aligned with industry- 
recognized standards, where available and appropriate.^ States use a variety of 
instruments to assess students’ technical skill attainment, including district- and state- 
developed assessments, standardized exams developed by public and private vendors, 
credentialing or certification exams developed by industry associations, or some 
combination of these approaches (U.S. Department of Education, 2008). States also use 
proxies for concentrators’ attainment of technical skills, including grade point average 
(GPA) in technical courses or completion of a CTE program sequence (Government 
Accountability Office [GAO], 2009). 

Of the 34 states using one or more technical assessments to meet Perkins IV 
accountability requirements in the 2006-2007 program year, the most commonly 
identified were associated with industry-developed certificates or credentials (32 states), 
followed by nationally developed exams (29 states; U.S. Department of Education, 

2008). Among states using nationally developed assessments not associated with an 
industry-developed certificate or credential, many contract with the National 
Occupational Competency Testing Institute (NOCTI), a non-profit corporation governed 
by a consortium of states; NOCTI provides standardized technical skill assessment 
development and scoring. 

Advocates for the use of technical skill assessments in CTE programs contend that testing 
will lead to educational improvements by clarifying instructional goals, raising 
performance expectations, and motivating students to achieve high-level standards 
(Bishop, 2002). Technical skills assessments also are envisioned to give state and local 
education agencies the information staff need to gauge program effectiveness and to 
direct assistance to programs that are performing inadequately (U.S. Congress Office of 
Technology Assessment, 1994). However, the usefulness of technical skills assessments 
for these purposes depends in part on whether the assessments indicate the attainment of 
desirable postsecondary outcomes. If students’ technical skill levels (as measured by 
technical skill assessments) are positively associated with post-program results, such as 
rates of employment, job performance and earnings, postsecondary enrollment, and 
degree completion, then program improvements might be directed to enhancing students’ 
skill acquisition. 



Background and Literature Review 
Technical Skills Assessments 

Over the past two decades, federal CTE policy has directed states to adopt strategies for 
tracking student attainment of technical skills. Although states retain considerable 
flexibility in how they assess concentrators’ technical skills, the U.S. Department of 



^ A concentrator is a student who completes a threshold level of CTE program coursework. In 
Pennsylvania, a secondary CTE concentrator is a student who, by the end of the reporting school year, has 
earned at least 50% of the minimum technical instructional hours required for PDE program approval. 
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Education, Office of Vocational and Adult Education (OVAE), has sought to narrow 
states’ reporting options. Non-regulatory guidanee — released by OVAE to assist states in 
designing measures for the Perkins IV 2006 reauthorization — ealls for states to use 
teehnieal skill assessments aligned with industry-reeognized standards, where available 
and appropriate, to measure CTE eoneentrators’ skill attainment (U.S. Department of 
Education, 2007). States lacking such assessments are directed to report on the estimated 
pereentage of students ineluded in the measure and their plan and timeframe for 
inereasing program and student eoverage. Despite these reeommendations, many states 
have expressed preferenees for the use of indireet measures of teehnieal skills, such as 
GPA or program completion. Although 44 states at the seeondary level and 33 at the 
postseeondary level initially agreed to use teehnieal skill assessments as measures of skill 
attainment, not all of these are expeeted to follow through (El.S. Department of 
Edueation, 2010). 

States introducing assessment programs have a range of options in seleeting teehnieal 
skill assessments. Some states (sueh as Georgia) are independently working to develop 
assessment systems based on their own state standards, whereas others are adopting 
national industry-reeognized assessments (Hyslop, 2009). In the latter case, states often 
default to exams reeognized by their own eredential ageneies, which typically cover 
oeeupations that may affeet the health or well-being of the publie, sueh as eosmetology or 
harboring (Klein & Chamer, 2006). Industry-reeognized exams are also widely 
employed, with states adopting oeoupation-speeific exams used to gauge students’ 
readiness to praetiee in the field. Eor example, many states have adopted National 
Automotive Teehnieians Edueation Eoundation (NATEE) exams to evaluate students 
eompleting studies in targeted areas, sueh as automobile serviee, eollision repair, and 
refinishing. Major software eompanies like Adobe, Microsoft, and Oracle also have 
developed eertifieation assessments to recognize student mastery of specifie design 
programs, sueh as Adobe ACE Maeromedia Elash, Mierosoft Certified SQE Server 
Administrator, and Oraele Eorms Developer Assoeiate (Southern Regional Edueation 
Board, 2009). 

Teehnieal skill assessments fall into two broad groups: general work readiness and 
oecupation-speeifie assessments. General work readiness assessments are intended to 
measure skills that may transfer aeross most oeeupations. These exams measure students’ 
knowledge of foundational work skills, such as the ability to follow directions, apply 
information, and work with others. In eontrast, oeeupation-speeifie exams test students’ 
mastery of teehnieal skills assoeiated with a parti eular job or narrow eareer area and often 
focus on assessing individuals’ knowledge and ability to apply advaneed eontent. 

To date, states implementing standardized teehnieal skill assessments at the high sehool 
level have relied primarily on narrowly foeused exams that assess student readiness for 
entry-level employment in a partieular job (Klein & Chamer, 2006). Recent federal 
efforts to promote the development of college and eareer readiness standards may affect 
the direetion in whieh these and other states’ assessment systems move. Moreover, as the 
eareer elusters movement gains momentum, states also will have the option of adopting 
foundation-level exams that cut across multiple oeeupations within a speeifie business or 
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industry field. 

Although not typically employed for federal reporting purposes, some states also are 
using act’s WorkKeys system to assess student attainment of career readiness skills. 
These include assessments to evaluate individuals’ applied job skills in communication, 
problem solving, and interpersonal relations, as well as personal skills relating to 
performance/behavior, talent, and potential organizational fit. 

One of the more popular testing options among states is provided by NOCTI. NOCTI 
was established as a consortium of states in the 1960s to coordinate research, test 
development, field-testing, and revision of occupational assessments. Before the 
establishment of NOCTI, states worked independently to develop occupational 
assessments; NOCTI coordinated and consolidated these efforts into its current 
organizational structure."^ The organization now offers roughly 170 standardized technical 
assessments, which are used in CTE programs in at least seven states. 

The NOCTI Job Ready skill assessments measure specialized, occupation-specific 
knowledge and skills associated with a particular job or narrow set of careers, such as 
architectural drafting or plumbing. The organization also offers a set of workplace 
readiness assessments that test students’ attainment of broad, crosscutting skills and 
knowledge associated with a range of occupations. These exams are used in Pennsylvania 
to assess students for whom a NOCTI occupation-specific exam does not exist. 

Linking Technical Skill Levels and Postsecondary Outcomes 

Technical skills are usually defined through the skill standards, or those abilities and 
knowledge necessary for competent performance in carrying out responsibilities 
associated with the workplace (Stone, 2009). As such, most research on technical skill 
assessments has focused on workplace outcomes. In the 1970s and 1980s, these studies 
centered on assessments as predictors of earnings and occupational competency in 
industry and the armed forces. Much of the research explored the association between 
individuals’ assessment scores and their likelihood of subsequent employment, earnings, 
and job performance, with some of the most exhaustive work performed by John Bishop. 
In particular. Bishop (1988) sought to assess the validity of the Armed Services 
Vocational Aptitude Bahery as a predictor of earnings and job performance in military 
jobs. Bishop’s analysis suggested that both the trade and technical subtests and the 
generic trade and technical competency tests of the battery are predictive of earnings for 
males, with less robust effects for other populations. Based on the generally positive 
associations found between technical assessments and labor market outcomes. Bishop 
noted that such assessments would have great potential for use in education settings, 
given their proven predictive value in the workplace (Bishop, 1993). 



^ Career clusters are used to represent a grouping of career specialties/occupations and broad industries 
sharing common knowledge and skills required for career success. For detailed information on the career 
clusters initiative, see http://www.careerclusters.org/ . 

^ For more information about the history of NOCTI, see A Brief History of NOCTI, 
http://www.nocti.org/History.cfm . 
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Bishop’s advocacy for the use of technieal skill assessments in edueation settings has 
found support in reeent federal and state policies requiring measures of technical skill 
attainment among CTE students (Meeder, 2008). These policies have emerged in the 
context of increased accountability and standards-based education reforms emphasizing 
the acquisition of academic skills among all students. In CTE, these reforms have 
resulted in efforts to prepare students for college and careers and increase the rigor of 
instruction in both areas. Attendant strategies inelude linking seeondary and 
postseeondary instruction, integrating aeademie skills in CTE eourses and programs, and 
developing career cluster programs to teach skills associated with a related set of 
occupations, rather than a specifie job (National Governors Assoeiation Center for Best 
Practices, 2007). 

As CTE programs have changed, new researeh has emerged examining the effeets of 
secondary level CTE partieipation on postsecondary outcomes. However, perhaps 
reflecting the reeent advent of many state assessment programs, these studies generally 
use CTE program partieipation and concentration or eompletion as measures of students’ 
CTE backgrounds. Most studies on workplace outcomes find positive effeets of 
secondary CTE programs on earnings, although most of these studies are on individuals 
who graduated from high school in the 1970s and 1980s (Altonji, 1994; Gray & Huang, 
1992; Kang & Bishop, 1986). Among those using more recent data. Bishop and Mane 
(2004) found that graduates who devoted one-sixth of their time in high school to CTE 
courses had higher earnings in the short and medium term than students who did not take 
any. 

Reflecting efforts to ensure that CTE students have the aeademie skills necessary to 
enroll and succeed in higher education, other researeh has addressed postsecondary 
education outcomes. Outcomes include the type of postsecondary institution in which a 
student enrolled (Eaird, Chen, & Eevesque, 2006) and degree or certificate completion 
(Eaird et al., 2006; Lekes et ah, 2007). Research has also investigated the assoeiation 
between CTE coursetaking at the secondary level and postsecondary enrollment, the 
focus of the present study. 

Two descriptive studies analyzing the postseeondary outcomes of 1992 high school 
graduates found that students who eombined CTE and college preparatory eourses were 
about as likely to enroll in postseeondary education as those who completed a eollege 
preparatory eurriculum alone (Hudson & Hurst, 1999; Eaird et al., 2006). By contrast, 
students who completed a CTE concentration alone enrolled in college at about half the 
rate of their peers in eollege prep programs. A study of career aeademies found 
traditional CTE concentrators to have a lower likelihood of college enrollment relative to 
students in career academies and college prep programs (Maxwell & Rubin, 2002). 

As with the studies on workforee outcomes, however, the data in these studies are on 
students who eompleted their secondary schooling before the policy changes of the mid- 
to late- 1990s. Using more reeent data, DeLuca, Plank, and Estacion (2006) analyzed 
transcript data from the 1997 National Eongitudinal Survey of Youth controlling for 
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student demographic and socioeconomic characteristics, high school GPA, engagement, 
and test scores. The study found that CTE coursetaking does not hinder college 
enrollment, but also that the higher the number of CTE relative to academic courses in a 
student’s transcript, the lower the likelihood that the student will enroll in college. 

These studies compare students in various CTE programs with students in college 
preparatory or academic programs, or programs that mix the two, with controls that 
include measures of academic achievement such as standardized math and reading test 
scores and GPA. CTE program experiences are differentiated only in terms of students’ 
overall curriculum (e.g., participation in a career academy) or the proportion of academic 
versus CTE courses completed. The effects of CTE program participation, however, may 
vary according to the level of technical skills that students acquire through their 
experiences in the program. In part, the skills associated with academic and workplace 
success may overlap, as suggested by recent exploratory research comparing college and 
career readiness assessments (ACT, 2006). This overlap may be especially true of CTE 
fields that require math skills, like information technology (IT). A study of career 
pathways programs found IT students more likely than their peers in other programs to 
take advanced math courses (Lekes et al, 2007). 

Technical skill proficiency may also be a proxy for a student’s ability or motivation. 
Students with relatively higher levels of technical skill may be more likely to enroll in 
postsecondary education to enhance their skills further. Moreover, labor market analyses 
indicate that occupations once requiring only a high school degree now require 
postsecondary training for entry as technology and computer use has expanded 
(Silverberg, Warner, Eong, & Goodwin, 2004). As more and more occupations require 
postsecondary training, highly skilled students may recognize that higher education in a 
technical field may enhance their opportunities when they do decide to enter the 
workforce. Traditionally, secondary CTE graduates were expected to transition to work 
rather than college; today, both the characteristics of the labor market and CTE program 
reforms may be making postsecondary education a more appealing and viable option. 
Technical skill levels may be an indicator of which students are more likely to follow this 
route. This study seeks to extend the research on the postsecondary outcomes of CTE 
graduates by exploring the association between the technical skill levels that CTE 
students attain and their likelihood of postsecondary enrollment. 

Research Design and Questions 

This research effort seeks to identify the characteristics of CTE concentrators in 
Pennsylvania who complete a NOCTI technical assessment and to assess the association 
between students’ technical skill levels and postsecondary outcomes. The analysis 
employs descriptive statistics to assess graduates’ employment results and logistic 
regression to examine the relationship between technical skill assessment scores and 
receipt of an industry-recognized credential on postsecondary enrollment, holding 
constant factors such as student gender, rigor of high school studies, and GPA. 

The dichotomous dependent variable in the multivariate analysis is whether a student 
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enrolled in higher education within a few years of high school graduation. Technical skill 
assessment score level and whether a student earned an industry certification are 
independent variables, and selected demographic and high school program variables 
serve as controls. Whether a concentrator earned an industry-recognized certificate is also 
included as an independent variable, because a growing number of states are using these 
certificates as measures of technical attainment (GAO, 2009). The logistic model is 
estimated separately for students taking occupation-specific and workplace readiness 
assessments and by CTE program area. 

The paper opens with an examination of whether CTE concentrators’ occupational 
competency assessment performance is related to their demographic characteristics and 
educational experiences during high school. Next, it assesses whether there is an 
association between technical skill levels and graduates’ participation in academically 
rigorous, upper-division math and science high school classes, self-reported overall GPA, 
or the earning of one or more industry certifications. The last set of analyses focuses on 
whether there is an association between NOCTI exam score levels and postsecondary 
enrollment and whether enrollment varies by high school CTE program completed. 

Data 

Analyses are based on Pennsylvania high school graduates who completed their CTE 
program course requirements and took a NOCTI technical assessment in the 2005-2006 
or 2006-2007 academic year, the most recent year for which follow-up data are available. 
The final dataset includes test scores produced by NOCTI; student demographic, 
educational transcript, and follow-up enrollment and employment data compiled from a 
dedicated CTE database maintained by the PDE; and postsecondary enrollment data 
obtained from the National Student Clearinghouse (NSC). 

The dataset was compiled in three stages. Eirst, student exam data were collected from 
NOCTI, which identified concentrators’ names, occupational competency exam score(s), 
and a set of demographic data (gathered when the test was administered) for each 
secondary student tested in 2005-2006 or 2006-2007. These data, corresponding to 
29,936 CTE concentrators, were matched with education record data maintained by the 
PDE, using concentrators’ names and birthdays as unique identifiers. Record matches 
provided data on student characteristics (age, gender, and race/ethnicity), transcript 
information (including CTE program and advanced academic courses completed), 
industry certifications earned, and employment outcomes. Matches were obtained for 
21,394 CTE graduates, or roughly 72% of students who were reported as taking a NOCTI 
exam in either 2005-2006 or 2006-2007. 




Table 1. Total number of Pennsylvania CTE completers who took technical skill assessments and for 
whom follow-up data are available In the current dataset; 2005-2006, 2006-2007, and 
2005-2006 and 2006-2007 combined 





2005- 


-2006 


2006- 


-2007 


2005-2006 and 
2006-2007 
combined 


Data 


No. 


% 


No. 


% 


No. 


% 


Number of student score records 
reported by NOCTf 


15,384 


100.0 


14,552 


100.0 


29,936 


100.0 


Number of secondary student records 
for whom NOCTI reported a test score 
and for whom a match was found in 
PDE CAT data (dataset total) 


11,083 


72.0 


10,311 


70.9 


21,394 


71.5 


Dataset total 


11,083 


100.0 


10,311 


100.0 


21,394 


100.0 


Number of records with an National 
Student Clearinghouse match 


4,564 


41.2 


4,284 


37.9 


8,848 


41.4 


Number/percent with a PDE 
follow-up survey 


3,911 


35.3 


2,217 


19.6 


6,128 


28.6 


Number/% with NSC match or 
PDE follow-up survey 


6,578 


59.4 


5,323 


47.1 


11,901 


55.6 


Number/% with NSC match and 
PDE follow-up survey 


1,897 


17.1 


1,178 


10.4 


3,075 


14.4 



^ Includes adult education students, which are excluded from the other rows. 

SOURCE: National Occupational Competency Testing Institute, National Student Clearinghouse, and the PDE. 



To collect post-program employment data, the PDE conducts an annual survey of CTE 
students who completed a CTE program either in a eomprehensive high school or at a 
CTE center and graduated the preceding academic year.^ This online survey requests 
reeipients to self-report on their enrollment and employment experiences in the second 
quarter following high sehool completion. Eor example, a CTE coneentrator graduating 
in June 2006 would report on his or her enrollment and employment outcomes for 
October 1-Deeember 31, 2006. Between 2005-2006 and 2006-2007, the PDE sent out a 
follow-up survey request to 21,213 CTE concentrators who graduated from high school. 
A total of 6,081 surveys were reeeived, whieh equates to a roughly 29% response rate for 
the two-year period. 

Postsecondary enrollment data were obtained by transmitting record identifiers for CTE 
eoncentrators to the NSC for matehing against the elearinghouse database.^ (A eomplete 



^ In 2004-2005 (the most recent year for which data are available), 67% of CTE completers attended area 
CTE centers (AVTS), and 33% attended a comprehensive high school (PDE, 2006). 

^ The NSC maintains data for over 3,300 institutions of higher education, enrolling 92% of all U.S. college 
students. Data include information on public and private colleges and universities, as well as many private 
trade schools. A listing of participating institutions can be downloaded from: 
http://www.studentclearinghouse.org/ . 
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list of variables and their sourees is ineluded in Appendix Table 1)7 For the analysis in 
this report, data were limited to students ages 19 or younger (i.e., adults participating in 
CTE coursework offered at a secondary institution were excluded). The final dataset 
produced by the NSC for postsecondary enrollment includes information for 21,228 
students, of whom 8,777, or roughly 41%, were found to be enrolled. 

Technical Skill Assessments in Pennsylvania 

The use of technical assessments in Pennsylvania predates the technical skill 
accountability requirements of Perkins IV. In 1996, the Pennsylvania State Board of 
Education approved the use of student occupational competency assessments for all 
secondary and adult CTE program concentrators scheduled to graduate (PDE, 2009). In 
the 2005-2006 and 2006-2007 academic years, an estimated 80% of Pennsylvania CTE 
program completers took a technical assessment near the end of their program. Students 
who were in programs with associated technical skills assessments were required to take 
the exam; students in other programs were issued waivers and could choose to take a 

g 

general workplace readiness exam. Among the students who took any assessment, nearly 
all were of NOCTI origin. 

In Pennsylvania, the majority of NOCTI assessments measure entry-level, job-ready 
knowledge and skills that are aligned with industry-validated CTE program standards 
(Munyofu, 2007). These exams are drafted by teams that include experts from industry 
and education who participate in drafting, piloting, and revising the assessments to ensure 
each test’s validity and reliability.^ High school students take the NOCTI exam in March, 
April, or May of their senior year, following the completion of the majority of their CTE 
program requirements. All of the NOCTI exams designed to test occupation-specific 
skills in Pennsylvania in 2005-2006 and 2006-2007 had both written and performance 
components that required examinees to demonstrate content knowledge and the ability to 
perform occupation-specific tasks. 

Students in CTE programs that were not associated with a NOCTI occupation-specific 
exam had the option to take a workplace readiness exam. Administered in written form 
(no performance component exists), the assessment gauges a spectrum of generic skills 



^ The complete appendix of tables is available on the NRCCTE website at: 

http://136.165.122.102/UserFiles/File/NOCTI PDE CTE/NOCTI PDE Appendix Tables FINAL.pdf 
* In 2005-2006, program completers were students who had completed CTE program requirements set at 
the school level. In 2007-2008, Pennsylvania phased out its written-only exams, including the general 
workplace readiness exam documented in this study. Currently, Pennsylvania requires that all technical 
assessments have a performance component (four exams in 2005-2006 and five in 2006-2007 had no 
performance component). Pennsylvania also phased out most of the technical skill assessments provided by 
other vendors, such that, as of the 2008-2009 program year, an estimated 95% of students completing CTE 
programs took a required end-of-program NOCTI exam (interview with Dr. Paul Munyofu, Research 
Associate, PDE Bureau of Career and Technical Education, July 20, 2009). 

^ For details on NOCTI’s test development process, see 
http://www.nocti.org/PDFs/DevelopmentRevisionataGlance.pdf 

*** The 2- to 3 -hour written test has about 200 multiple choice items, and the 3- to 4-hour performance test 
requires the test taker to complete a series of tasks associated with two to seven occupation-specific jobs or 
activities (Munyofu & Kohr, 2009). 
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applicable to most industries and eareer areas. The exam includes questions to assess 
students’ capaeity to eommunieate and solve problems, as well as their understanding of 
teamwork and leadership abilities. 

Technical Skill Levels 

The Bureau of Career and Teehnieal Edueation at the PDE sets cut points that determine 
four score levels. The written exam eut points that determine the levels are reassessed 
roughly every two years, eorresponding to when NOCTI assessments are revised and 
updated. Results for the performanee portion of the assessment are evaluated by industry 
praetitioners using eriteria in the assessment’s evaluator guide provided by NOCTI. The 
eut scores for this portion of the assessments are uniform aeross all exam types. 

Pennsylvania has established a system for establishing eriterion-referenced benchmarks 
for the written exams using the Nedelsky method (Munyofu & Kohr, 2009). The cut 
scores are set by 12- to 15-member panels composed of CTE teaehers and industry 
representatives. Panel judges individually review eaeh test item and identify the answers 
that a minimally eompetent test taker would be able to reeognize as obviously wrong 
before resorting to guessing on the remaining ehoiees. The mean of the panel judges’ 
scores becomes the pereent eut score for the test.^* 

These resulting levels are: 

• Advanced Level: Refleets mastery of competenee and understanding of 
aeademie/career and teehnieal skills and knowledge required for advanced 
plaeement in employment and/or postseeondary edueation. 

• Competent Level: Refleets a solid aequisition of aeademic/eareer and 
technical skills and knowledge required to enter employment and/or 
postsecondary education. 

• Basic Level: Reflects an adequate attainment of aeademie/career and teehnieal 
skills and knowledge required to enter employment or postseeondary 
edueation. Students with this seore -would funetion at an entry level, but 
would require some assistanee on the job.” 

• Below Basic Level: Reflects a partial acquisition of skills and knowledge 

needed to perform a given assignment, task, or operation on the job. 

Additional instruetion and/or assistance are necessary in order for the student 

to suecessfully eomplete speeific assignments. Students with this seore have 

12 

not acquired the minimum skills required for an occupation. 

In addition to course requirements, to complete a CTE program, students must aehieve a 
score at or above the basic level. Students seoring at the two highest levels earn 
certifieates: the Pennsylvania Competeney Certifieate, if their overall score is at the 
eompetent level, and the Pennsylvania Skills Certifieate, if they score at the advaneed 



*' For detailed descriptions of how the cut scores are determined, see Munyofu, 2008, and Munyofu and 
Kohr, 2009. 

See http://www.pde.state.pa.us/career edu/lib/career edu/2008 Certification Guide.pdf . 
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level. Aside from measuring skill attainment, occupational competency exam results have 
other uses, including the identification of low-performing districts by the PDE. The state 
and districts negotiate yearly improvement goals, of which student performance on 
occupational competency assessments represents one component. Students may also use 
their occupational competency results to earn postsecondary credits (or a waiver of 
program requirements) in accordance with agreements between individual high schools 

13 

or districts and postsecondary institutions. 

Results 

The next sections first present descriptive statistics on the percentage of CTE completers 
who took occupation-specific and workplace readiness assessments, by technical skill 
level, and the percentage of CTE graduates who took an assessment, by CTE program 
area and technical skill level. The subsequent section compares technical skill levels by 
type of CTE program completed, gender, race/ethnicity, the number of advanced math 
and science courses passed, and self-reported GPA. The final set of descriptive statistics 
compares the technical skill levels of students who earned and did not earn one or more 
industry certifications.^"^ The concluding sections present logistic regression analyses 
predicting the effects of technical skill levels on postsecondary educational enrollment. 

Descriptive Statistics 

Skill level by exam type. The 2005-2006 and 2006-2007 test taker data include test 
scores for 85 different exams. Overall, nearly 86% of students completed an occupation- 
specific exam at some point during the two-year study period, indicating that the majority 
of Pennsylvania high school CTE curricular offerings are perceived as aligned with 
NOCTI assessments.^^ Students for whom a NOCTI exam does not exist took a 
workplace readiness exam. 



Interview with Dr. Paul Munyofu, Research Associate, PDE Bureau of Career and Technical Education, 
July 20, 2009. 

Comparisons reported in the text were tested using Pearson’s chi-square test for independence and are 
significant at the /? < .05 level. 

For occupation-specific exams, the largest group of assessments administered was in Cosmetology (8% 
of exams), followed by Food Production and Management, Carpentry, and Early Childhood Care (about 
4% each). See Appendix Table 2 for the percentage of students taking specific exams overall and Appendix 
Table 3 for test-taking rates by CTE program area. 
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Table 2. Percentage of GTE graduates who took occupation-specific and workplace readiness assessments, by technical 
skill level: 2005-2006 and 2006-2007 





2005- 


-2006 and 2006-2007 






2005- 


-2006 






2006- 


-2007 






Occupation- 

specific 


Workplace 

readiness 


Occupation- 

specific 


Workplace 

readiness 


Occupation- 

specific 


Workplace 

readiness 


Skill Level 


No. 


% 


No. 


% 


No. 


% 


No. 


% 


No. 


% 


No. 


% 


Below Basic 


4,569 


25.1 


545 


17.8 


2,195 


23.9 


340 


18.4 


2,374 


26.4 


205 


16.9 


Basic 


3,501 


19.3 


549 


18.0 


1,826 


19.9 


321 


17.4 


1,675 


18.6 


228 


18.8 


Competent 


2,479 


13.6 


386 


12.6 


1,310 


14.3 


253 


13.7 


1,169 


13.0 


133 


11.0 


Advanced 


7,623 


41.9 


1,576 


51.6 


3,834 


41.8 


929 


50.4 


3,789 


42.1 


647 


53.3 


Percentage 

by 

assessment 

type 


18,172 


85.6 


3,056 


14.4 


9,165 


83.3 


1,843 


16.7 


9,007 


88.1 


1,213 


11.9 



SOURCE: National Occupational Competency Testing Institute and PDE, Bureau of Career and Technical Education. 



Students were relatively suecessful in their assessment efforts across the two program 
years. As detailed in Table 2, a majority scored at the advanced level on both the 
workplace readiness and occupation-specific assessments. However, a greater proportion 
of students taking workplace readiness exams (52%) achieved the advanced performance 
level than those taking occupation-specific (42%) exams. This may reflect the more 
challenging nature of the career-specific exams, which require that students demonstrate 
an advanced set of job-focused skills within a focused area of study. Although relatively 
similar proportions of students scored at the competent and basic score levels on both 
types of assessments, a larger percentage of students taking occupation-specific exams 
scored at the below basic level compared to those taking workplace readiness 
assessments (25% vs. 18%, respectively). 

Skill level by CTE program area. The PDE categorizes NOCTI exams by program type 
for reporting purposes (PDE, 2006). Over the two program years, a majority of students 
completed coursework in the Trade and Industrial Education program area (54%), 
followed by Occupational Home Economics Education (12%) and Business 
Education/Marketing and Distributive Education (11%).'^ 



This analysis combines the — Bsiness Education” and ^Marketing and Distributive Education” groups. 
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Table 3a. Percentage distribution of all CTE graduates who took either type of assessment and those 
who took a workplace ready assessment, by CTE program area: 2005-2007 



Any NOCTI assessment Workplace readiness assessment 



Program Area 


No. 


Percentage 

distribution 


No. 


% 


Percentage 

distribution 


Agriculture Education 


1,026 


4.8 


108 


10.5 


3.5 


Business Education/ 
Marketing and 
Distributive Education 


2,363 


11.1 


133 


5.6 


4.4 


Health Occupations 
Education 


2,058 


9.6 


344 


16.7 


11.3 


Occupational Home 
Economics Education 


2,535 


11.9 


49 


1.9 


1.6 


Trade and Industrial 
Education 


11,478 


53.7 


523 


4.6 


17.1 


Other (not elsewhere 
classified)^ 


1,934 


9.0 


1,899 


98.2 


62.1 



^ Includes career exploration/awareness skills, diversified occupations, and vocational education, other. 

SOURCE: National Occupational Competency Testing Institute and PDE, Bureau of Career and Technical Education. 



Students enrolling in CTE coursework that cannot be classified into a specific program 
area or those concentrating in programs for which a NOCTI occupation-specific exam is 
not available are tested using a workplace readiness exam. As illustrated in Table 3a, 
roughly 98% of students taking other CTE coursework (i.e., could not be classified) 
completed such a workplace readiness assessment. Students who could be classified into 
a program area, but for whom a NOCTI occupation-specific exam did not exist also take 
a workplace readiness exam. Among these students, the largest population was taking 
coursework in the health occupations (17%), followed by agriculture (11%). 

Student performance on NOCTI assessments varies by program area. The percentage of 
students scoring at the advanced level was highest in the health occupations for students 
taking both occupation-specific (61%) and workplace readiness (60%) exams. Generally, 
skill levels were different for workplace readiness exams and occupation-specific exams, 
with overall scores higher for workplace readiness than occupation-specific assessments. 
The exceptions are health occupations and occupational home economics, for which the 
association between exam type and score level was not significant {p < . 01). 
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Table 3b. Percentage distribution of CTE graduates who took NOCTI assessments, by CTE program area and technical 
skill level: 2005-2006 and 2006-2007 (combined) 

Program Area 





Agriculture 


Business/ 
Marketing & 
Distribution 


Health 

Occupations 


Occupational 

Home 

Economics 


Trade and 
Industrial 


Other^ 


Skill Level 


No. % 


No. % 


No. % 


No. % 


No. % 


No. % 



Occupation-specific 

Below 



Basic 


334 


36.4 


757 


34.0 


190 


11.1 


476 


20.5 


2,797 


25.5 


15 


42.9 


Basic 


161 


17.5 


384 


17.2 


266 


15.5 


424 


18.3 


2,258 


20.6 


8 


22.9 


Competent 


113 


12.3 


285 


12.8 


216 


12.6 


306 


13.2 


1,558 


14.2 


1 


2.9 


Advanced 


310 


33.8 


804 


36.1 


1,042 


60.8 


1,114 


48.0 


4,342 


39.6 


11 


31.4 



Workplace readiness 

Below 



Basic 


18 


16.7 


7 


5.3 


21 


6.1 


16 


32.7 


117 


22.4 


366 


19.3 


Basic 


25 


23.1 


18 


13.5 


59 


17.2 


9 


18.4 


88 


16.8 


350 


18.4 


Competent 


14 


13.0 


15 


11.3 


57 


16.6 


9 


18.4 


70 


13.4 


221 


11.6 


Advanced 


51 


47.2 


93 


69.9 


207 


60.2 


15 


30.6 


248 


47.4 


962 


50.7 



^ Includes career exploratlon/awareness skills, diversified occupations, and vocational education, other. 

SOURCE: National Occupational Competency Testing Institute and PDE, Bureau of Career and Technical Education. 



Students taking CTE coursework in the other (not elsewhere elassified) eategory had the 
lowest pereentage of results at the advaneed or eompetent levels on an oeeupation- 
speeifie exam. This result might be expeeted if these students took an ooeupation-speoifie 
exam for whieh they were not adequately prepared. However, beeause most students in 
this situation took workplaee readiness exams, it is diffieult to infer mueh from this 
finding. Indeed, students not elsewhere elassified performed relatively highly on 
workplaee readiness skills, suggesting that a majority of these students were given a solid 
foundation of work preparation skills. 

Skill level by demographic characteristics. To assess the effeet of gender and 
raee/ethnieity on student performanee, researehers disaggregated NOCTI exam results. 

As illustrated in Table 4, more students took ooeupation-speoifie rather than workplaee 
readiness exams, although a higher pereentage of males than females did so (89% vs. 
82%, respeotively). 
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Table 4. Percentage distribution of CTE graduates who took a NOCTI exam, by technical skill level and gender and 
race/ethnicity: 2005-2007 

Gender Race/Ethnicity 



Female Male Asian Black Hispanic White 



Skill Level 


No. 


% 


No. 


% 


No. 


% 


No. 


% 


No. 


% 


No. 


% 


Total 


9,388 


44.9 


11,528 


55.1 


143 


0.7 


1,193 


5.6 


830 


3.9 


19,030 


89.8 


Occupation- 


specific 


7,712 


82.1 


10,199 


88.5 


117 


81.8 


1,024 


85.8 


665 


80.1 


16,333 


85.8 


Workplace 


readiness 


1,676 


17.9 


1,329 


11.5 


26 


18.2 


169 


14.2 


165 


19.9 


2,697 


19.9 


Occupation- 


specific 


7,712 


100.0 


10,199 


100.0 


117 


100.0 


1,024 


100.0 


665 


100.0 


16,333 


100.0 


Below Basic 


1,588 


20.6 


2,918 


28.6 


30 


25.6 


411 


40.1 


188 


28.3 


3,928 


24.1 


Basic 


1,548 


20.1 


1,904 


18.7 


25 


21.4 


190 


18.6 


142 


21.4 


3,136 


19.2 


Competent 


1,100 


14.3 


1,346 


13.2 


15 


12.8 


118 


11.5 


101 


15.2 


2,242 


13.7 


Advanced 


3,476 


45.1 


4,031 


39.5 


47 


40.2 


305 


29.8 


234 


35.2 


7,027 


43.0 



Workplace 



readiness 


1,676 


100.0 


1,329 


100.0 


26 


100.0 


169 


100.0 


165 


100.0 


2,697 


100.0 


Below Basic 


222 


13.3 


313 


23.6 


4 


20.0 


47 


27.8 


49 


29.7 


443 


16.4 


Basic 


286 


17.1 


251 


18.9 


4 


20.0 


43 


25.4 


31 


18.8 


469 


17.4 


Competent 


239 


14.3 


144 


10.8 


9 


15.0 


26 


15.4 


17 


10.3 


340 


12.6 


Advanced 


929 


55.4 


621 


46.7 


9 


45.0 


53 


31.4 


68 


41.2 


1,445 


53.6 



SOURCE: National Occupational Competency Testing Institute and PDE, Bureau of Career and Technical Education. 



The largest percentages of males and females scored at the advanced level, relative to the 
percentages who scored at the lower levels on both types of exams, with higher 
percentages of females than males at this level (45% vs. 40% for occupation-specilic 
exams, and 55% versus 47% for workplace readiness exams, respectively). Fewer 
females than males also scored at the below basic level for both exams. 

A majority of NOCTI test takers were White (90%), followed by Black (6%), Hispanic 
(4%), and Asian (1%). Over 80% of the students in all racial/ethnic groups took an 
occupation-specilic exam, though higher percentages of Whites and Blacks took these 
exams (86%) than Hispanics (80%) and Asians (82%). 

With the exception of Black students, the largest proportion of students in all racial/ethnic 
groups scored at the advanced level on occupation-specific exams, with 43% of White, 
40% of Asian, and 35% of Hispanic students achieving advanced status. A higher 
percentage of Black students scored at the below basic than at the advanced level (40% 
vs. 30%, respectively). 

Skill level by course rigor and high school grade point average. The PDE maintains 
data on all advanced courses passed by each CTE student above the minimum required 

1 n 

for graduation. Past research on student transcripts has found advanced math and 



The Bureau of Career and Technical Education maintains a subset of academic and program level data 
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science courses to be strongly associated with postsecondary attendance, grades, and 
degree completion (Adelman, 1999, 2006). The analysis therefore excludes non-math and 
science courses. Algebra I/IB and Biology I, Applied Geometry, Anatomy and 
Physiology, and the level I courses of other math and science sequences. The number of 
the remaining courses passed, which include Advanced Placement math and science 
courses. Algebra II, and Trigonometry, were totaled for each student. CTE graduates 
were classified into three categories based on the number of upper-division math and 
science courses they passed; none, one to two courses, and three or more courses. Table 5 
summarizes student coursetaking patterns and technical skill levels. 



Table 5. Percentage distributions of CTE graduates' technical skill levels, by exam type and the 
number of advanced math and science courses passed: 2005-2007 





No 


1-2 


3 or more 




courses 


courses 


courses 


skill Level 


No. % 


No. % 


No. % 



Occupation-specific 



Below Basic 


2,040 


32.2 


1,760 


23.0 


769 


18.3 


Basic 


1,317 


20.8 


1,572 


20.6 


612 


14.6 


Competent 


848 


13.4 


1,081 


14.1 


550 


13.1 


Advanced 


2,123 


33.6 


3,236 


42.3 


2,264 


54.0 


Total 


6,328 


100.0 


7,649 


100.0 


4,195 


100.0 


Workplace readiness 


Below Basic 


237 


27.8 


215 


18.3 


93 


9.1 


Basic 


208 


24.4 


212 


18.0 


129 


12.6 


Competent 


103 


12.1 


170 


14.5 


113 


11.0 


Advanced 


306 


35.8 


578 


49.2 


692 


67.4 


Total 


854 


100.0 


1,175 


100.0 


1,027 


100.0 



^ Includes career exploration/awareness skills, diversified occupations, and vocationai education, other. 

SOURCE: Nationai Occupational Competency Testing Institute and PDE, Bureau of Career and Technical Education. 



Irrespective of exam type, students taking more upper-division academic coursework 
scored at the highest levels on NOCTI tests. For example, over one-half (54%) of 
students who passed three or more upper-division math and science courses achieved the 
advanced level on a NOCTI occupation-specific exam, as compared with just over one- 
third (34%) of those who did not pass an upper-division course. These findings may 
indicate that students completing relatively more upper-division academic courses 
possess cognitive abilities or personal motivation that enable them to perform at higher 
levels on NOCTI exams than those lacking such personal characteristics. 

Students taking a technical skill assessment are asked to include their high school GPA in 
a short questionnaire administered with the written test. About 15% reported an A 
average, 52% a B average, 31% a C average, and 2.2% an average grade of D or lower. 



for CTE students, including academic coursetaking. See Appendix Table 4 for a complete list of the 
academic courses included in the PDE student file. Courses in the secondary category (upper-division 
advanced math and science courses) are marked with an asterisk in the Appendix Table. 
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Table 6 summarizes students’ exam seores by their self-reported GPA and eombines the 
two lowest eategories into a single eategory for C or lower. 



Table 6. Percentage distributions of students' technical skill levels, by assessment type and self- 
reported average high school GPA: 2005-2006 and 2006-2007 (combined) 



Average GPA of C or 





Average GPA of A 


Average GPA of B 


lower 




skill Level 


No. 


% 


No. 


% 


No. 


% 


Occupation-specific 


Below Basic 


517 


18.8 


2,165 


24.0 


1,740 


29.9 


Basic 


453 


16.5 


1,711 


19.0 


1,232 


21.1 


Competent 


339 


12.3 


1,224 


13.6 


812 


13.9 


Advanced 


1,445 


52.5 


3,925 


43.5 


2,044 


35.1 


Total 


2,754 


100.0 


9,025 


100.0 


5,828 


100.0 


Workplace readiness 


Below Basic 


44 


10.5 


247 


15.2 


249 


25.2 


Basic 


60 


14.3 


283 


17.4 


204 


20.7 


Competent 


31 


7.4 


236 


14.5 


115 


11.6 


Advanced 


284 


67.8 


859 


52.9 


420 


42.5 


Total 


419 


100.0 


1,625 


100.0 


988 


100.0 



^ Includes career exploratlon/awareness skills, diversified occupations, and vocational education, other. 

SOURCE: National Occupational Competency Testing Institute and PDE, Bureau of Career and Technical Education. 



The results reveal a positive assoeiation between GPA and teehnieal assessment score {p 
< .01). The majority of students reporting a GPA of A scored at the advanced level on the 
occupation-specific and workplace readiness exams (53% and 68%, respectively). The 
lowest percentages of students with scores at the advanced level are found among those 
who reported a GPA of C or lower. This group of students also had the highest 
percentages of students scoring at the basic and below basic levels. 

Industry certifications. Pennsylvania secondary GTE students may earn industry- 
recognized certifications during high school. To assist local school districts in identifying 
certifications for which their students may be eligible, the PDE publishes a guide that 
lists known industry-recognized certifications by GTE program (PDE, 2008a). Eees for 
certification tests are covered by individual school districts, which help to ensure that 
students who elect to take an exam have completed the necessary preparatory coursework 
to succeed. About 10% of the students in the current dataset had earned an industry 
certification during high school. Just over half of these students (58%) earned one 
certification, and another 26% had two or three. Table 7a summarizes the percentage of 
students in each GTE program category that earned a certification. The highest 
percentage of students earning an industry certification was in health occupations 
education (33%), followed by occupational home economics education (19%). 



** For a complete list of certifications, and the number of certifications earned by certification type, see 
Appendix Table 5. 
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Table 7a. Percentage of secondary CTE graduates who took a NOCTI technical skills assessment 
who earned an industry certification, by CTE program area: 2005-2006 and 2006-2007 
(combined) 






No. of 


Earned an industry 
certification 


Program Area 


students 


No. 


% 


Total 


21,394 


2,145 


10.0 


Agriculture Education 


1,026 


36 


3.5 


Business Education/Marketing and Distributive Education 


2,363 


151 


6.4 


Health Occupations Education 


2,058 


679 


33.0 


Occupational Home Economics Education 


2,535 


488 


19.3 


Trade and Industrial Education 


11,478 


782 


6.8 


Other (not elsewhere classified)^ 


1,934 


9 


0.5 



^ Includes career exploration/awareness skills, diversified occupations, and vocational education, other. 
SOURCE: PDE, Bureau of Career and Technical Education. 



These findings may reflect the relatively higher proportion of industry certifications 
available in some programs, rather than the success of the program itself. For example, 
the most popular certifications in 2005-2006 and 2006-2007 were issued by the 
American Red Cross, and hence catalogued into the Health Occupations area. The most 
common certification was Infant, Child, and Adult CPR (cardiopulmonary resuscitation) 
(12% of all industry certifications earned), followed by First Aid (9%), and Adult 
CPR/AED (cardiopulmonary resuscitation/automated external defibrillators; 8%). 

Although certification tests often cover a far narrower range of skills than NOCTI 
occupation-specific assessments, a higher percentage of students earning one or more 
industry certification than students who did not earn a certification scored at the advanced 
level on both the workplace readiness and occupation-specific exams. Certifications 
may therefore indicate skill or motivation levels similar to those found among students 
who score at a high level on an occupational competency assessment. 



It also may be that students have greater use for some assessments. For example, the relative popularity 
of American Red Cross exams may reflect students seeking certification so that they can pursue babysitting 
employment. 

Interview with Dr. Paul Munyofu, Research Associate, PDE Bureau of Career and Technical Education, 
July 20, 2009. 
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Table 7b. Percentage of secondary CTE graduates who took a NOCTI technical skills assessment 

who earned an industry certification, by technical skill level: 2005-2006 and 2006-2007 

(combined) 



Earned an industry certification 



Yes No 



skill Level 


No. 


% 


No. 


% 


Occupation-specific assessment score level 


Below Basic 


240 


12.4 


4,391 


26.7 


Basic 


308 


15.9 


3,240 


19.7 


Competent 


261 


13.5 


2,249 


13.7 


Advanced 


1,132 


58.3 


6,598 


40.0 




1,941 


100.0 


16,478 


100.0 


Workplace readiness assessment score level 


Below Basic 


14 


7.2 


536 


18.5 


Basic 


33 


17.0 


526 


18.2 


Competent 


32 


16.5 


358 


12.4 


Advanced 


115 


59.3 


1,477 


51.0 



^ Inciudes career exploration/awareness skills, diversified occupations, and vocational education, other. 
SOURCE: PDE, Bureau of Career and Technical Education. 



Although students who earned a certifieation represent just a tenth of the test takers 
included in the current dataset, their numbers are expected to grow. The Bureau of Career 
and Technical Education is working to increase the number of students earning industry 
certifications/ credentials by 30% by 2010, a goal that may reflect national trends. Other 
states are also encouraging students to earn industry certifications; Florida, for example, 
recently passed an act requiring industry-recognized certifications for CTE program 
completion at the secondary and postsecondary levels. 

Postsecondary employment. Students who earn a relatively high score on a NOCTI 
occupation-specific or workplace readiness assessment might be expected to possess a set 
of skills that beher prepare them for post-program success. This may be manifested by a 
higher likelihood of finding employment immediately following high school. To assess 
the post-program outcomes of CTE graduates, researchers matched student performance 
data on the NOCTI exams with information collected by the PDE follow-up survey and 
maintained at the NSC. 

The PDE collects data on the employment status of all CTE program graduates in the 
second quarter (i.e., September 1-December 31) following their high school completion. 
Information is collected using an online survey instrument, which asks individuals to 
self-report if they were employed and/or enrolled at any point during the period. State 



http://www.portal.state.pa.us/portal/server.pt/communitv/bureau of career technical education/7334 . 
Florida Career and Professional Education Act (CAPE) of 2007. 

Because students are not asked to provide information on their employment field, length of experience, 
or status (i.e., full-time or part-time), it is not possible to gauge whether their post-program experiences are 
related to their in-school training. 
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records indicate that a total of 21,213 survey requests were sent out to CTE concentrators 
in the 2005-2006 and 2006-2007 program years. Of these surveys, a total of 6,081 (32%) 
were returned with usable information. 



Analysis of missing data indicates that the lower a graduate’s technical assessment score, 
the lower the likelihood that they would return a follow-up survey: nearly one-third 
(32%) of students scoring at the advanced level responded to the survey request, 
compared with less than one-quarter (23%) of those at the below basic level. This result 
might be expected if student performance is associated with post-program work or 
college placement; those failing to find work or enroll in college might be less motivated 
to report on their negative post-high school experiences (Baj, Trott, & Stevens, 1991).^"^ 



Table 8a. Number and percentage distribution of CTE graduates who took NOCTI exams and 

returned a follow-up survey, by technical skill level and reported postsecondary status: 
2005-2006 and 2006-2007 (combined) 



Technical Skill Level 



Below 



Follow-up Survey 


Total 


Advanced 


% 


Competent 


% 


Basic 


% 


Basic 


% 


PDE follow-up survey^ 


21,213 


9,189 


100.0 


2,863 


100.0 


4,049 


100.0 


5,112 


100.0 


Respondent 


6,081 


2,978 


32.4 


864 


30.2 


1,062 


26.2 


1,177 


23.0 


Non-respondents 


15,136 


6,214 


67.6 


1,999 


69.8 


2,987 


73.8 


3,936 


77.0 



Occupation-specific 



assessment 


5,185 


2,462 


100.0 


739 


100.0 


929 


100.0 


1,055 


100.0 


Enrolled only 


1,359 


667 


27.1 


164 


22.2 


217 


23.4 


311 


29.5 


Enrolled and employed 


1,725 


898 


36.5 


253 


34.2 


289 


31.1 


285 


27.0 


Employed only 


1,746 


769 


31.2 


285 


38.6 


348 


37.5 


344 


32.6 


Unemployed (not 
enrolled) 


355 


128 


5.2 


37 


5.0 


75 


8.1 


115 


10.9 



Workplace readiness 



assessment 


896 


516 


100.0 


125 


100.0 


133 


100.0 


122 


100.0 


Enrolled only 


200 


133 


25.8 


27 


21.6 


22 


16.5 


18 


14.8 


Enrolled and employed 


421 


246 


47.7 


61 


48.8 


66 


49.6 


48 


39.3 


Employed only 


242 


127 


24.6 


30 


24.0 


37 


27.8 


48 


39.3 


Unemployed (not 
enrolled) 


33 


10 


1.9 


7 


5.6 


8 


6.0 


8 


6.6 



^ Detail presented here does not sum to the total of 8,848 NSC matches found and the 6,128 PDE surveys because 
comparable exam scores were not possible to calculate for all assessments. 

NOTE: Excludes students over 19 and students who did not graduate in the year they took the test. 

SOURCE: National Occupational Competency Testing Institute and PDE, Bureau of Career and Technical Education. 



Tests of association were conducted between technical skill levels, assessment types, and 
employment and education status as reported in the follow-up survey. Technical skills 



Specifically, when assessing the feasibility of using Unemployment Insurance wage record data to assess 
Job Training Partnership Act (JTPA) programs, Baj et al. (1991) found that 70.2% of those employed at 
program termination responded to their survey request versus 49.6% of those unemployed. 
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levels are positively assoeiated with education status for both types of assessments, and 
for employment status for occupation-specific assessments only. The results did not 
indicate that the relationship between technical skill levels and employment or education 
status differed between the two types of assessment. This result may reflect the relatively 
small proportion of graduates that responded to the survey, particularly among those who 
took a workplace readiness assessment. 

Postsecondary enrollment. Identifiers for CTE graduates taking NOCTI assessments 
were matched against records maintained by the NSC to identify students who attended 
college at any time between their high school graduation and the fall of 2009, the time at 
which the match occurred. A total of 21,228 records were shared with the NSC, and a 
total of 8,777 matches (41%) achieved. These data likely underestimate the actual 
enrollment of Pennsylvania CTE graduates, because it is possible that there was a 
discrepancy between the name and birth date listed for a student in the PDE and those 
maintained by the NSC. It is also possible that an individual attended a postsecondary 
institution not included in the NSC database. Although mismatches due to name and 
birth date discrepancies are unlikely to be correlated with technical skill assessments, 
those due to students enrolling in an institution not reporting to the NSC could bias 
results if unaffiliated institutions were more likely to enroll students with lower test 
scores. 



Table 8b. Number and percentage distribution of CTE graduates who took NOCTI assessments, by 

technical skill level and postsecondary enrollment status: 2005-2006 and 2006-2007 (combined) 

Technical Skill Level 



Below 



Follow-up Survey 


Total 


Advanced 


% 


Competent 


% 


Basic 


% 


Basic 


% 


National Student 
Clearinghouse (NSC)^ 


21,228 


9,199 


100.0 


2,865 


100.0 


4,050 


100.0 


5,114 


100.0 


Enrolled 


8,777 


4,446 


48.3 


1,201 


41.9 


1,441 


35.6 


1,689 


33.0 


Not enrolled 


12,451 


4,753 


51.7 


1,664 


58.1 


2,609 


64.4 


3,425 


67.0 


Occupation-specific 


18,172 


7,623 


100.0 


2,479 


100.0 


3,501 


100.0 


4,569 


100.0 


Enrolled 


7,155 


3,471 


45.5 


998 


40.3 


1,186 


33.9 


1,500 


32.8 


Not enrolled 


11,017 


4,152 


54.5 


1,481 


59.7 


2,315 


66.1 


3,069 


67.2 


Workplace readiness 


3,056 


1,576 


100.0 


386 


100.0 


549 


100.0 


545 


100.0 


Enrolled 


1,622 


975 


61.9 


203 


52.6 


255 


46.4 


189 


34.7 


Not enrolled 


1,434 


601 


38.1 


183 


47.4 


294 


53.6 


356 


65.3 



SOURCE: National Occupational Competency Testing Institute, National Student Clearinghouse, and the PDE. 



Because detailed information on enrollment status was not available, it is neither possible to determine 
the type of institution an individual attended (i.e., two-year or four-year college or university, trade school, 
etc.) or the length of time for which a student was in attendance. 

Some 94% of headcount enrollments in Pennsylvania were included in the NSC in 2005, but percentages 
for other states ranged from as high as 99% to as low as 67% (National Center for Higher Education 
Management Systems, 2005). 
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Overall, enrollment in a postseeondary institution was positively eorrelated with NOCTI 
test scores, and this finding held irrespective of whether students took a workplace 
readiness or occupation-specific exam. However, a higher percentage of students with 
scores at or above the basic competency level on a workplace readiness assessment 
enrolled in postseeondary education than those taking occupation-specific assessments. 
For example, 62% of students scoring at the advanced level on a workplace readiness 
exam were found to be enrolled at some point in the study period, compared with 46% of 
those scoring at the advanced level on an occupation-specific exam. Individuals scoring 
at the below basic level were least likely to be found enrolled, and relatively little 
difference was found between those scoring below basic on different types of 
assessments. 

Multivariate Analysis 

The multivariate analyses assess the association between a high school CTE graduate’s 
technical skill attainment level (as measured by a technical competency exam score) and 
postseeondary enrollment. The analyses use sequential logistic regression to estimate a 
set of nested binary logistic regression models of postseeondary enrollment among 
NOCTI occupation-specific and workplace readiness test takers. The outcome is whether 
a student enrolled in postseeondary education (as measured by an NSC match). Low 
survey response rates and the relatively small number of respondents reporting 
unemployment status precluded the estimation of models using employment measures as 
dependent variables. 

Table 9 summarizes descriptive statistics for the variables in the estimated equation for 
students who took an occupation-specific exam and students who took a workplace 
readiness exam. The dichotomous dependent variable indicates whether a student 
enrolled in one of the institutions of higher education included in the NSC at any time 
between high school graduations in 2005-2006 or 2006-2007 through the 2009 calendar 
year. Students identified as enrolled may or may not have been seeking a degree, and 
may have been enrolled at any point during the period covering from when they 
graduated high school and the time that study measurements were made in the fall of 
2009. 



Because one more year of data is available for 2005-2006 than 2006-2007 graduates, the data may 
underestimate higher education enrollment for 2005-2006 graduates relative to 2006-2007 graduates. A 
Pearson’s chi-square test of independence (/? < .01) for graduation year and higher education enrollment 
was not significant, and the variable for graduation year was not significant (/? < .01) when included in the 
full logit model (results available upon request from the authors). 
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Table 9. Descriptive statistics for variables used in the analysis of the effect of NOCTI occupation- 
specific assessment score level on postsecondary enrollment 



Variable 


Occupation- 

Specific 


Workplace 

Readiness 


Enrolled in postsecondary education 


anytime between graduation (2005-2006 or 2006-2007) and 2009 


0.39 


0.53 


NOCTI assessment score level 


Advanced 


0.42 


0.52 


Competent 


0.15 


0.13 


Basic 


0.19 


0.18 


Below Basic^ 


0.25 


0.18 


Earned an industry certification 


0.11 


0.06 


Control Variables 


Female 


0.43 


0.56 


Ethnicity 


Asian 


0.06 


0.66 


Elispanic 


0.06 


0.06 


Black 


0.04 


0.05 


White^ 


0.90 


0.88 


Advanced (upper-division) STEM courses passed 


3 or more courses 


0.35 


0.34 


1-2 courses 


0.42 


0.38 


None^ 


0.23 


0.28 


Self-reported grade point average 


A 


0.16 


0.14 


B 


0.51 


0.54 


C or lower^ 


0.33 


0.33 



^ Reference category. 

SOURCE: National Occupational Competency Testing Institute, National Student Clearinghouse, and the PDE. 



The independent variables are dummy variables for a student’s overall teehnical 
assessment score level (advanced, competent, basic, and below basic as the reference 
category), and whether a student earned an industry certification. The control variables 
are factors associated with postsecondary enrollment, including gender, race/ethnicity 
(reference category: White), passage of advanced math and science courses during high 
school (reference category: none), and self-reported GPA (reference category: C or 
lower). 

Table 10 presents the results of several logistic regression models estimated for graduates 
that took an occupation-specific NOCTI assessment. The models evaluate the effects of 
students’ technical skill levels in a specific occupational area, and earning an industry 
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certification on postsecondary enrollment while controlling for student demographic and 
academic background characteristics. Model 1 includes just the demographic variables 
(gender and race/ethnicity). Model 2 includes demographic characteristics and secondary 
academic background variables (passage of advanced science and math courses and self- 
reported GPA). Model 3 adds the two measures of technical skill achievement: whether a 
student earned an industry certification and technical assessment score level. 



The results are presented as odd ratios (for coefficients in terms of log odds, see Appendix Table 6). 
Odds ratios are calculated by taking the natural antilogarithms of the logit coefficients. Values exceeding 1 
indicate that students in a particular group (e.g., students scoring at the advanced level on a technical skills 
assessment) have higher odds of enrolling in a postsecondary institution than the comparison group or 
omitted category (e.g., students who scored at the below basic level) and values less than 1 indicate lower 
odds. 
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Table 10. Logit models of postsecondary enrollment among NOCTI occupation-specific test takers: 2005-2006 and 2006-2007 





Model 1 






Model 2 






Model 3 




odds Ratio 


SE 


95% Cl 


Odds Ratio 


SE 


95% Cl 


Odds Ratio 


SE 


95% Cl 



Ethnicity^ 






















Asian 


2.91 


• .575 


1.97-4.28 


2.17 


* .446 


1.47-3.24 


2.21 


* 


.457 


1.47-3.31 


Black 


1.58 


• .103 


1.38-1.79 


1.74 


* .120 


1.52-1.99 


1.84 


* 


.128 


1.60-2.10 


Elispanic 


1.20 


.098 


1.03-1.41 


1.37 


* .116 


1.16-1.62 


1.41 


* 


.121 


1.19-1.66 


Gender (Female^l) 


1.55 


• .048 


1.45-1.64 


1.47 


* .047 


1.37-1.56 


1.45 


* 


.048 


1.36-1.55 


Advanced (upper-division) STEM courses passed^ 






















3 or more courses 








4.05 


* .178 


3.71-4.41 


3.83 


* 


.171 


3.51-4.18 


1-2 courses 








1.59 


* .060 


1.47-1.71 


1.54 


* 


.059 


1.42-1.66 


Self-reported grade point average^ 






















A 








1.57 


* .078 


1.42-1.73 


1.50 


* 


.076 


1.36-1.66 


B 








1.31 


* .048 


1.22-1.41 


1.29 


* 


.048 


1.20-1.38 


NOCTI assessment score level"* 






















Advanced 














1.39 


* 


.059 


1.28-1.51 


Competent 














1.28 


* 


.071 


1.15-1.43 


Basic 














1.00 




.051 


.91-1.11 


Earned an industry 
certification 














1.05 




.056 


.95-1.17 


Likelihood ratio Chi-squared 




293.92 






1,555.34 








1,638.51 




n= 




17,703 






17,483 








17,319 





* Significant at the p < .01 level. ^Omitted category: White. ^ Omitted category: No advanced STEM courses. ^ Omitted category: C or lower. “^Omitted category: Below Basic. NOTE: Odds ratios are calculated by taking the natural 
antilogarithms of the logit coefficients to ease interpretation. Values exceeding 1 indicate increased odds of enrolling in postsecondary education, and values less than 1 indicate decreased odds. SOURCE: National Occupational 
Competency Testing Institute and PDE, Bureau of Career and Technical Education. 
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The results of Model 3 suggest that students’ teehnieal skill levels are positively 
associated with higher education enrollment when controlling for a number of factors that 
have been found to influence higher education enrollment. The results suggest that, when 
compared to students who scored at the below basic level (the reference category), those 
at the competent and advanced levels were more likely to enroll in a postsecondary 
education institution within two to three years of graduation. By contrast, the odds ratio 
for earning an industry certification was not significant, indicating that those who earn an 
industry certification were no more likely to enroll than those without a certification after 
controlling for exam score. The results from Model 3 indicate that after controlling for 
race/ethnicity, gender, advanced science and math coursetaking, and GPA, the odds of an 
individual who scored at the competent level enrolling in higher education are greater by 
a factor of 1 .27 than the odds for a student with a score at the below basic level. For those 
with scores at the advanced level, the odds of enrolling in a postsecondary education 
institution are greater by a factor of 1.39 than the odds of those with scores below the 
basic level. 

The effects of race/ethnicity and gender in Model 1 indicate that the odds of enrolling are 
higher for Asians than Whites (the omitted category), and higher for women than men, 
findings which echo previous research (DiPrete & Buchman, 2006; Kao & Thompson, 
2003). However, the odds of enrolling for Hispanics and Blacks are higher than those for 
Whites in Models 2 and 3, a finding that contradicts a number of studies on race and 
higher education enrollment (Kao & Thompson, 2003). This finding may be related to the 
limits of the sample (CTE high school graduates) or the outcome measure, which 
includes any postsecondary enrollment and not just those seeking a degree. These 
variables, and the measures of high school academic background added in Model 2, are 
significant in Model 3 and change little in magnitude with the addition of assessment 
score level and industry certification variables in Model 3. 

Table 1 1 summarizes the results of the same models for the smaller group of about 3,000 
high school CTE graduates who took a workplace readiness assessment. Model 3 
includes the measures of technical skill while controlling for secondary academic 
background and demographic characteristics. As was found among students who took 
occupation-specific assessments, the odds of postsecondary education enrollment for 
students who scored at the competent and advanced levels are higher by factors of 1.70 
and 2.22, respectively, than the odds of a student scoring at the below basic level. In 
contrast to the results for occupation-specific test takers, scoring at the basic level, 
relative to the below basic level, is also associated with postsecondary enrollment. The 
odds of enrolling for students scoring at the basic level are 1 .42 time those scoring at the 
below basic level, net of the included academic background and demographic 
characteristics. 
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Table 11. Logit models of postsecondary enrollment among NOCTI workplace readiness test takers: 2005-2006 and 2006-2007 





Model 1 






Model 2 






Model 3 




odds Ratio 


SE 


95% Cl 


Odds Ratio 


SE 


95% Cl 


Odds Ratio 


SE 


95% Cl 



Ethnicity^ 






















Asian 


1.09 


.536 


.41-2.86 


0.94 


.476 


.35-2.54 


1.05 




.543 


.38-2.89 


Black 


1.44 


.240 


1.04-1.99 


1.59 


* .272 


1.14-2.23 


1.83 




.319 


1.30-2.58 


Elispanic 


1.19 


.196 


.86-1.64 


1.30 


.222 


.93-1.82 


1.43 




.248 


1.02-2.01 


Gender (Female^l) 


1.89 


• .141 


1.64-2.19 


1.82 


* .141 


1.56-2.12 


1.70 




.134 


1.46-1.99 


Advanced (upper-division) STEM courses passed^ 






















3 or more courses 








2.68 


* .269 


2.20-3.26 


2.27 




.236 


1.86-2.79 


1-2 courses 








1.16 


.109 


.97-1.40 


1.07 




.102 


.89-1.29 


Self-reported grade point average^ 






















A 








2.00 


* .257 


1.55-2.57 


1.81 




.237 


1.40-2.34 


B 








1.24 


.106 


1.05-1.47 


1.17 




.102 


.99-1.39 


NOCTI assessment score level"* 






















Advanced 














2.22 




.249 


1.78-2.76 


Competent 














1.70 




.245 


1.28-2.26 


Basic 














1.42 




.185 


1.10-1.83 


Earned an industry 
certification 














1.40 




.231 


1.01-1.93 


Likelihood ratio Chi-squared 




80.31 






249.75 








311.59 




n= 




2,988 






2,973 








2,973 





* Significant at the p < .01 level. ^Omitted category: White. ^Omitted category: No advanced STEM courses. ^ Omitted category: C or lower. ^Omitted category: Belov\/ Basic. 

+ Excluded from the analysis due to small cell sizes (less than 5). NOTE: Odds ratios are calculated by taking the natural antilogarithms of the logit coefficients to ease interpretation. Values exceeding 1 indicate increased odds of 
enrolling in postsecondary education, and values less than 1 indicate decreased odds. SOURCE: National Occupational Competency Testing Institute and PDE, Bureau of Career and Technical Education. 
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Earning an industry certification is not significantly related to the odds of enrolling in 
higher education in any of the models tested for students who took oecupation-specific or 
workplaee readiness assessments. As indicated in Appendix Table A5, about half of the 
certifieates earned were for specifie skills or sets of skills assoeiated with the health 
professions, such as CPR or first aid. Although more information is needed, it may be 
that the certifications earned are associated with a number of occupations, some of whieh 
allow entry with a high sehool degree and others that require postsecondary training. It 
also may be that only certain types of eertifications (such as certifications that test a range 
of skills) are assoeiated with higher edueation attendanee. The low percentage of students 
who earned an industry eertifieation (about 10%) and the small number of different 
certifications earned mean that sample sizes are too small to explore outeomes by 
eertifieation type. 

The highest odds ratios among all the variables ineluded in the model for students who 
took an oecupation-speeific assessment (but not in the model for students who took a 
workplaee readiness assessment) are those assoeiated with passing three or more 
advanced math and seience courses. The results from Model 3 in both sets of models 
indicate that, after eontrolling for technical skill level, eertifications, race/ethnicity, 
gender, and GPA, the odds of enrolling in higher education for a student who passed 
three or more advanced math and science eourses are higher by a faetor of 3.83 (among 
oecupation-specifie test takers) and 2.27 (among workplaee readiness test takers) than the 
odds for a student who did not pass any of these eourses. The direction and magnitude of 
these findings are consistent with previous research that found math and science eourses 
to be among the strongest predictors of postsecondary enrollment and completion (ACT, 
2009; Adelman, 2006; Miller, 2007). 

As noted earlier, the test takers eompleted a variety of CTE programs and studied 
subjects as diverse as forestry, web page design, and dental assisting. It might be 
expeeted, given the different courses, skills, and employment and postseeondary 
education options associated with these programs, that the assoeiation between teehnieal 
assessment score level and higher edueation would vary depending on the type of 
program in whieh the student enrolled. Appendix Tables 7. 1-7.5 include descriptive 
statistics and the results of logistie regression models estimated for eaeh of five subjeet- 
defmed program areas: agriculture edueation; business/marketing and distributive 
education; health occupations education; occupational home economies education; and 
trade and industrial education.^^ 

Overall, results similar to those found for the whole sample were found for eaeh of the 
CTE program areas. The exceptions were agrieulture edueation and occupational home 
economics education. Eor agriculture students (Appendix Table 8.1a), scores at the basic 
and advanced levels (relative to the below basic level) were positively assoeiated with 
postseeondary enrollment relative to scoring at the below basic level, whereas a score at 
the competent level was not, when demographic and secondary schooling variables were 



The -ether (not elsewhere classified)” category - which includes career exploration/awareness skills, 
diversified occupations, and vocational education, other - was not analyzed separately because a subject or 
occupational area does not unite the programs included in it. 
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held constant. The results for students who completed occupational home economics 
programs (Appendix Table 7.4a) indicate that scoring at any of the three passing levels 
(basic, competent, or advanced) increased the odds of postsecondary enrollment relative 
to scoring at the below basic level. 



Conclusion 

This study seeks to identify links between CTE concentrators’ performance on NOCTI 
occupational competency assessments administered in Pennsylvania and their 
postsecondary education and early labor market experiences. In examining the extent to 
which students’ test scores on these two types of assessments are associated with their 
post-program experiences, this study seeks to gauge the predictive power of occupation- 
specific and workplace readiness exams. 

In designing the study, researchers selected Pennsylvania due to its commitment to 
technical skill assessment and the robustness of its education data: Today, nearly every 
student in the state who completes a CTE program sequence takes a standardized, state- 
recognized NOCTI competency assessment, and in the years examined in the study, over 
80% of CTE program sequence completers took this assessment. Moreover, student 
employment follow-up data, collected annually using a standardized, online survey 
instrument, offers consistent information on the post-program workforce experiences of 
CTE concentrators graduating from a program. Einally, the state’s subscription to the 
NSC supports the collection of postsecondary enrollment data for students entering 
public and private colleges and universities located across the country. 

Data Quality 

In attempting to assemble the database, researchers encountered a number of challenges 
that complicate statistical analysis. Although the state administered nearly 30,000 NOCTI 
exams to high school students completing a CTE program sequence across the 2005- 
2006 and 2006-2007 program years, PDE administrators were able to match test records 
with less than three-quarters (72%) of students’ educational records. Although the 
unmatched records were likely random in nature, resulting from differences in how 
student names and other identifiers are recorded on NOCTI exams and represented in 
PDE administrative databases, the absence of more than one-quarter of student records 
calls into question whether study results are generalizable to all CTE completers. Eow 
response rates on the statewide survey of CTE concentrators, in particular for students 
scoring at the two lowest score levels, also may overstate the post-program enrollment 
and employment outcomes of CTE concentrators. 

Pennsylvania is taking steps to improve the match rate for students completing NOCTI 
exams by creating a unique student test identifier that will be incorporated into student 
education records. Although this will likely improve future data quality, challenges 
associated with creating a usable database for this study underscore that the state has not, 
to date, taken full advantage of the information contained in NOCTI test reports. 
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Measures of Technical Skill and Postsecondary Enrollment 

Among the students for whom reeord matehes were aehieved, deseriptive statisties 
indieate that those who seore at the highest levels on a teehnieal skills assessment were 
more likely to have taken a workplaee readiness exam, to be female, and to be enrolled in 
a health oeeupations or oeeupational home eeonomies program. Higher seoring test takers 
also were more likely to have eompleted three or more advaneed math and seienee 
eourses. 

Multivariate analyses indieate that, when eontrolling for raee/ethnieity, gender, advaneed 
seienee and math eoursetaking, and GPA, the odds of enrolling in higher edueation for a 
student who seored at the advaneed level on an ooeupation-speoifie exam are greater by a 
faetor of 1.39, and on a workplaee readiness exam by a faetor of 2.22 than the odds for a 
student with a seore at the below basie level. For those with seores at the eompetent level, 
the odds of enrolling in a postseeondary edueation institution are greater by a faetor of 
1.28 for students taking an oeeupation-speeifie exam and by a faetor of 1.70 for students 
taking a workplaee readiness exam than for those with seores at the below basie level. No 
assoeiation was found between earning an industry eertifieation and postseeondary 
attendanee. 

The highest odds ratios are those assoeiated with students passing advaneed math and 
seienee eourses. The results indieate that, after eontrolling for teehnieal skill level, 
eertifieations, raee/ethnieity, gender, and GPA, the odds of enrolling in higher edueation 
for a student who passed three or more advaneed math and seienee eourses are higher by 
a faetor of 3.83 for those taking an oeeupation-speeifie exam, and by a faetor of 2.27 for 
those taking a workplaee readiness exam than the odds for a student who did not pass any 
of these eourses. This suggests that GTE eoneentrators taking both advaneed aeademie 
and oeeupationally foeused eoursework may benefit from their studies. 

Future Research 

With additional data, the analysis presented here might be expanded in several direetions 
and offer new insights into the assoeiation between students’ teehnieal skill attainment 
(as measured by teehnieal skill assessments) and both postseeondary edueational and 
labor market outeomes. 

Reeent ehanges in how Pennsylvania state data are eolleeted have inereased the amount 
of seore information that Pennsylvania obtains from NOCTI. In 2008, the PDE arranged 
with NOCTI to report all test seores assoeiated with a partieular ID number. The PDE 
provides funding to loeal distriets for one teehnieal assessment for eaeh CTE eompleter 
or eoneentrator. In the aeademie years ineluded in this study (2005-2006 and 2006- 
2007), only the seore on the last teehnieal skill assessment a student took was available 
for analysis. However, distriets may also pay for a written pre-assessment, whieh is 
designed to be administered at the beginning of a program and assist in doeumenting 
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program gains. The PDE has developed a program to support CTE programs by using 
pre-assessments to identify students in need of extra assistanee or remediation. About 
half of the students taking pre-assessments take them in the spring of their junior year, 
and the rest in the fall of their senior year.^^ The expanded data will indieate the extent to 
whieh distriets use this option, and how this praetiee impaets student seores and 
outeomes. The use of a unique identifier rather than sehool and student names may also 
inerease the number of NOCTI student reeords that ean be matehed to PDE data. 

The opportunities for exploring the assoeiation between teehnieal skill eertifieations and 
postseeondary outeomes are also likely to inerease over time. The PDE Bureau of Career 
and Teehnieal Education is working to increase the number of certifications earned by 
CTE secondary students. The list of recommended industry certifications also includes a 
number of certifications that measure skill sets (PDE, 2008). The list for health sciences, 
for example, includes BLS Health-care Provider, issued by the American Heart 
Association, which includes skills related to a number of health-care emergencies, and 
includes CPR and AED. Should the number of students earning these types of industry 
certifications increase, it would be worth exploring whether a student earning an industry 
certification associated with a particular set of skills is more likely to be employed or to 
enroll in higher education than a student with a more limited or no certification. 

The high school curriculum program data used in this study were from the PDE’s Bureau 
of Career and Technical Education’s Secondary Career and Technical Education 
Information System (CATS) data system, and GPA was self-reported by the student 
when the NOCTI assessment was administered. Although the CATS system has detailed 
demographic data for students, the high school program data are more limited; for this 
study, the relevant variables were advanced courses passed and technical certifications 
earned. Einking these data with the state’s K-12 data system could potentially provide 
more detailed measures, including cumulative high school GPA, individual course 
grades, and more detailed transcript information such as CTE courses taken. 

Additional data on students’ high school experiences would contribute to our 
understanding of the influence of program and school factors. CTE students in 
Pennsylvania attend courses either in comprehensive high schools offering CTE 
programs or in area CTE centers. Moreover, Pennsylvania CTE students have the 
opportunity to participate in tech prep and diversified occupation programs. Participation 
in specialized CTE programs has been shown to influence the likelihood of students 
entering postseeondary education (DeLuca et ah, 2006) and may also affect technical 
skill levels. 

A natural extension of this study would be to examine the relationship between technical 
skills and postseeondary outcomes with the additional postseeondary educational 
outcomes including first-year and cumulative GPA and graduation (Geiser & Santelices, 
2007). Research on the postseeondary educational experiences of secondary CTE 
concentrators suggests, however, that outcomes for CTE students may differ from those 
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associated with students who pursued a college preparatory track in high school. Recent 
researeh on the postseeondary transcripts of CTE concentrators indicates that they are 
more likely to attend a subbaeealaureate institution and to earn fewer postsecondary 
credits overall but more in career-related (CTE) courses than college preparatory 
coneentrators (DeLuca et ah, 2006; Laird et ah, 2006). The findings suggest a number of 
other postsecondary outcomes, ineluding type of institution attended, type of degree or 
certifieate earned, and number of credits earned in CTE and other courses, that may be 
assoeiated with a student’s score on a teehnical assessment. Also of interest might be 
whether a student pursues a postsecondary program in a field related to their secondary 
CTE program. 

Although postsecondary educational outcomes are inereasing in importance as a measure 
of CTE program effectiveness, labor market outcomes are also of interest. Two outeomes 
that have been used in a large number of studies on postsecondary labor market outcomes 
include employment status and earnings (Griffith & Wade, 2001; Rose, 2006; Silverberg 
et ah, 2004). Euture research may focus on quantifying the assoeiation between students’ 
performanee on teehnical skill assessments, at both the high school and postsecondary 
levels, and their subsequent post-program employment results. Given the challenges 
assoeiated with collecting valid employment data using traditional survey efforts, states 
may wish to eonsider identifying alternative sources of employment data — for example, 
by matching student records with state unemployment insurance reeords. 
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